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ABSTRACT . 

The attempt by a small number of school districts to 
forge a linkage among* testing and evaluation and instructional 
decision making is examined. The sample districts (approximately six) 
exhibit characteristics that represent the diversity of American 
school districts in size, locale, and student demographics. Thrjbe 
researchers spent approximately 1 week in eiach district visiting 
schools and district offices, interviewing district participants,, and 
examining relevant document* and records. Distrifct respondents were 
asked about three general areas: (1) Why is this district trying to 
link, testing and evaluation with instructional decision-making? (2) 
How does this district do this? and (3) What effects have the linking 
activities had? The following three specific questions are discussed 
as they relate to the first two areas of interest:^ (1) What are the^ 
incentives and disincentives that operate in school districts 
attempting to forge an evaluation-testing-instruction linkage? ( 2) 
What are examples of the approaches districts are taking to forge 
these linkages? and (3) What are the potential contributions this 
research has for school improvement? (PN) 
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In July 1979, we began a three-yea? inquiry to discover ways in 

which school, districts might effectively link their district testing 

and evaluation activities with instructional decision making/ 

**•«•• . • 

This inquiry was stimulated by our belief, based on previous re- 

search and experience in school districts, that testi-ng and ^evaluation 
activities in. most districts had only* limited influence on internal 
school district instructional decision making. Instead, the focus of 
testing and evaluation in many districts seemed to be toward satisfying 
external demands, 'e.g., federal program evaluation requirements, court- 
ordered desegregation mandates (Zucker, 1981;^David, 1978). But many 
school districts had move'd to^develop their testing and evaluation ca- 
pacities .[Lyon, et al . , T978) and it seemed logical to us that the 
data and reports generated by a district evaluation unit might also 
serve as a district curriculum and instructional management information 
system. 

The main purpose of our work is not to determine the extent to which 
a nationwide sample of school districts are using testing and evaluation 
for internal instructions^ decision making. Instead, we are examining 
tjow a small number of districts are attempting to forge a linkage among 

testing and evaluation and instructional decision making. • ■ 

»■ -i 

At y the present time, we haVe completed extensive case studies in 
five or/six districts that we selected because they had a reputation^ 
for having tried to forge this linkage. Our sample districts, while not 
Comprising a national sample, do exhibit characteristics that represent ; 
iversi^y of American school^ district!. They reflect differences 1 in: 



size (large/smafl), student demographies (affluent/below-average in- 
come, racially homogeneous/racially heterogenous) , and locale (urban/ 
.suburban). Three^researchers have 'each spe.nt approximately one week 
in each district visiting schools and district offices, interviewing 
district participants, examining relevant documents and recorefs. We 
have asked respondants about three general areas: Why is this , district 
trying to link testing and evaluation with instructional decjsiyon-making?. 
How" does this district do this? What effects have the linking activities 
had? ' . \ 

In the brief space available to us, we would like to discuss three 
specific questions related only to the first two areas of interest. 

1, What are the incentives and disincentives'" that operate in 
school districts attempting to forj&e, an evaluation-testing- 

? instruction, linkage? ■ ( T 

2. -What are examples of the approaches ^districts are taking 



to forge these linkages 



? 



•3. What are the potential contributions this research has for 

school improvement? ' 
But before doing so, we'd like to » define briefly what we mean by 
linkage although you will get its fuller flavor./b^example, later in . 
the paper.' Linkage, to us, means the coordinatidn— either through formal 

9 " / 

or informal means--of all the operations and services within a school 
district essential or supportive of the use of testing and evaluation for 
instructional- purposes.. Linkage is, a function of management. It.is- an 
arrangement which brings together in some productive manner data collec- 
tion, analysis, and reporting with core instructional activities. 



* Such testing-evaluation-instruction linkages are not commonplace in 

school districts although testing and evaluation activ-itiek have increased. 

substantially since 1965\ > This may mean that most school districts have, 

over the past 15 years, felt little need to make such a linkage. We were 

interested, to learn what factors seemed to be encouraging our samplje dis- 

tricts to move in this direction, / ^ 

Question 1. What are the incentives and* disincentives that operate" 

in school districts attempting to forge an evaluation- 

9 testihg'instruction linkage? 

In the districts we studied, the single shared reasfen given for irli- . 

tiating coordination arrangements between tests and evaluations was to in-- 

fluence pupil achievement as measured on test scores. In many of the dis-. 

tricts there had been expressed-dissatisfaction, coming from a number of 

sources, with the academic performance of students. The move towards use 

of tests and evaluation data was primarily remedial, 'In one of the dis- 

• • *i 

tricts, however, there had been overall satisfaction with student learning; 

moreover, there was a sense, on the part of the district superintendent, 
that individualized instruction might increase the learning of average and 
above average students. 

.District officials indicated in their interviews with us that their 
overall intention was tQ use test scores w as a description of student achieve 
ment. They wanted these scores arranged and understood in such a way so 
as to redirect instruction. However, the immediate incentives for starting 
and continuing such a process seemed to vary from district to district. For 
example, some central offices were moved in this direction by explicit man- 
date from courts, or from state legislatures or" from school boards. In 
other districts, superintendents or other officials seemingly influenced , # 



by research and current educational thinking^ decided to use available 
federal and state money to build instruciionally relevant tests. 

We might category ze the types of incentives we found as either 
"sticks 11 or "carrots" and their sources as either externa.! or internal 
to district management. Our matrix would look something like this.' 



insert Figure 1 about here 
* : 



This list^o/ incentives, to some^extent, b'egs thfe question. The 
carrots and th? sticks are common to pther districts. Why haven't they. 
, moved to link testing and evaluation with instruction? Given our small 
sample, and our field-based research design, we" cannot provide a gpneral 
answer to. that question. What we can say is that certain characteristics 
seem tQ be present in our five districts, especially those that are most 
advanced in their linkage development. These elements indicate that our 
districts had the management capacity to respond to the incentives. The 
elements we refer to are: idea champions,* stable cote staff , realistic 
problem analysis, and .tolerance* for ambiguity. The following is * brief 
description of each element: 

° Idea champions—by this we mean individuals in key adminis-, 
tr^tive and policy positions who firmly believe, in the value 
- , of test and evaluation data and consistently championMts 

development and connection to instruction. In our districts, 
these individuals were found in a variety of positions. 
There was no consistent pattern, to their school district 
assignments, e.g., some are in curriculum, some in ev&lua- 
tion, some are line administrators; what they do share with' 



Sticks . ,.'\ • Carrots 



— - — — ,» ^ 1 

Requirements by federal or stat4. 
agencies to: • 
. °evaluate problems 
°develop courses* of s£udy 
°raise test scores 
°community dissatisfaction with 
public education expressed byt 
°press/and media 
°loss of students 
- ° Board action 


• Availability of fed&ral and. state 
money for 

•text development ^ r 
°evaluation of. programs 
° staff development f 
^relationships with universities 
- 0 existence of techniques br 
t procedures to link, tests with' - 
' ^instruction 


°decision of district administra- 
tors to link testing, evaluations, 
and instruction , % 


<v . * 
°desire of district to acquire" 
additional funds 
° presumed likelihood of success 
in linking tk$tin$ and evaluation 
< r \with education 



Figure, 1 



one another are characteristics such as conviction, per- 
suasiveness, and some degree of power. 

Stabljs core group— in our districts, these '"idea champions" 
and their followers have been around.for a while. In our 
most advanced linkage systems it has taken from 8-10 years 
for the linkage programs to develop and mature . . This ,could 
not ijaye happened ff tj3e core group had continually changed./ 
Comprehensive rather»than ad hoc problem analysis— the core 
group h^s been aware of next steps beyond the immediate task 
or the moment. It* is one thing to develop, foi 1 * example, a 
CRT program in reading—it is quite another thing to actually 
get teachers to us.e it* Bridging the gap between develop- 
ment anTt use implies an understanding *of the school site and 
district as- a bureaucratic social system and an appreciation 
of the various strategies ctnd tools that might most effectively 
Dridge the gap. \ 

< : ( \ ^ • \ 

Toleraice for ambiguity— none of the linkage arrangements 
developed, over time, iin anything resembling the rational, 
linear way tha^ is often described in standard planning * 
texts, instead, the programs (lave developed unevenly,* com- \ 
ponent by component on a broken front. Many tirnes, the 
components -of the linking system have been developed indepen- 
dently of one ^another, with different purposed and each with 
its own set of advocates-. Developing linkage arrangements to 
merge together these disparate' pieces into a new configuration 
takes time and it can be very frustrating. 



1 * * 

The above list is not exhaustive and it may be that these charac- 
ten'stics and activities are found in many districts that have not. 
thought about or who have tried and abandoned an effort to develop a 
linkage system; -.we don't know. ^ All we can-say is that these are^pre- 
sent in our ^sample districts and we believe they contribute to the pro- 
gress these districts have made. ^ N X 

* Whajt about disincentives to develop a .management arrangement that* 
links service? and supports^to tl^connection between testing, Evaluation 
and instruction? As we indicated above, typically districts are not 
ihg in this'direction. There are^^clely sfeveral reasons for this^ % 



movi 

An important one, we believe, is that fchese districts are not pr'essured* 

V pulled to think about the impact of stuc%ts^jtest scores through 

change in instructional- activities., Thus,-they continue in a traditional 

arrangement of semi -autonomous operational* untts. 

* < ■ " .- / * 

For example, districts may feel that their declining test scores 

are caused by -large and rapid changes in the ethnic or racial class 

make-up of their pupil population o.r that their declining performance 

merely reflects the flagging public support for the schoojs. They rea- 

son that, until these conditions change, it 0 is unlikely that ^encouraging 

< * . 

curriculum and instructional changes based on.test scores and evaluation 
findings will make much of a difference. They conclude that other poli- 
tical, social, or financial Wrategies .might be more appropriate. 

Another disincentive is that a closely linked testing, evaluation 
and instructional system, with its eftphasis on ^supervision, communica- 
tion, and coordination* flies in the face, of the traditional school dis- 
trict operating mode which can be characterized as loosely coupled, 
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(Weick, 1976) with teachers working quke independently behind 'closed 
classroom doors v (Lo>tie, 1975).. Teachers do not readily embraceap- 
proaches that fundamentally alter their accustomed ^professional ^behavior 

pattern's . * , 

Still another disincentive may be that a tight and interactive rela- 
tionship between test scores and classroom practice, is yet an unproven 
solution to the problem of student learning. While various components, 
e.g.,' development and use of CRTs, formative and summat^Ve evaluation 
methods, -are becoming increasingly sophisticated and! technically sound,, 
much remains to be done^before teacher's and administrators are convinced 
that these techniques can be used as effective tools in their own class- 
rooms "for improving student Achievement. Given the oth^r demands on their 
t^ime and energy, teachers will not readily eornmit themselves to unknown v 
and "'ut^r^/en technologies. * , 

Questiork.2. What are examples of the approaches p 

nets, are, taking "to forge these linkages? 
r samplSiis^?S£sa^e using a variety of approaches in linking 
testing andj€fval^|^'with instructional improvement. 1 fi In this paper 
we will very bfrrctfly describe two approaches: a decentralized, school - 
oriented systerrusing norm-referenced standardize^ test, scores ;"and . 
a tiistrict-dTrected centralized system using^ district-directed crjteri on- 
referenced- tests'. The decentralized NRT system usef the individual 
school as the locus of change. Within loosely prescribed district para- 
meters, each school has considerable discretion in developing and imple- - % 
meniting an instructional program that the school staff feels is appro- 
priate for its particular studefit body. The norm-referenced' student test 
results are foldedjnto'' an individualized evaluation report that i^-prepared 
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for each'school by the district office. The school staff, with the las^ls- 
tance of the central evaluation unit staff antf, often, with, district in- • 

i • ' 

<( struetional and curricular specialists^ develop "yearly plans in which 

• .they identify their own instructional -and other programmatic priorities. 
7 % ? ' Presumably, Revaluation unit's reports, including the scores, form & 
1 part of the evidence upon which each individual school modifies its in- 
struetional program. Some of these districts werealso developing and 
using CRTs, but these ^tests' d^d not play a prominent part in, their in- p > 
struetional renewa> program; they were used more as an instructional 

tool in the classroom rather than as a tool for school -site decision- 

' ■* • - ■ f 

making * ' . - -\ 

The school districts using: a centralized CRT system focus on a 
common district instructional continuum, usually'' in reading, math, and 

' • " i 

language arts to t which all schools are expected to adhere. The impetus 
for change comes more:fronr the district level than from the local- 4 
school. The district also encourages' the teaching staff to follow a 
comjioh instructional methodology when implementing the district's^cur- 
riculum. * Student scores on XRTs are used. as the'm#in fcasfs upon which 
instructional effectiveness is- gauged. The CRTs are developed s°o tjhat^ 
they relate to the district's adopted instructional program, NRTs are 
administered and reviewed but they are used mainly to .inform the public 
of the (district's program—they do not play a - prominent part in the • 
instructional renewal program. 

We do not wish to imply thai NRTs are not appropriate for ctectffc 
%*a1ized system? or that CRTs are ' inappropriate for decentralized sys- 
terns. We are merely reporting that these^ere the confirmations we v 
observed in<SSf small sampl^of districts/* Likely other mixtures of; 

y^c - ... ' ' .12'., ' . - v „ 



these elements have been devised. 

' * 0 f ' \ - 

Although the decentralized and centralized orientations differ in 

• • * • . *\\ • • A 

the locus of change and the types of test that are used, the districts 1 
arrangements share important characteristics such as providing support 
services to the schools: e.g. ty an extensive and appropriate in-service 
component, a well-developed data processing capability, a skilled evalu- 
ator and measurement staff. 

% The districts differ in regard to what they considered the effect 
of their programs. The two centralized, CRT-system districts pointed to „ 
what they considered substantial improvements in pupil achievement as a 
result of their program.-' The decentralized districts were less sure of 
the overall effect of their program on student 1 achievement but cited pro- 
cess changes at the school in evidence of effect; This is understandable 
since "the schools themselves differ in what they are trying to accomplish; 
and these diverse intentions do not lend themselves to more standard yard- 
sticks of progress. Of course, it may be that it takes longer to see the 
effects of a decentralized program than a more centralized one. We are not 
yet prepared to offer reasons for, or to assess the differences in the ef^ 
fectiveness of th^ two approaches? The districts themselves were not pre- 
sently examining what might be considered unintended or unexpected side 
effects,^ e.g. , heightened or lowered teacher morale, increased or de- 
creased community support. 

■ o ; 

. " Question 3. What are the potential contributions this research 
has for school improvement? 
There is a substantial public and professional "crisis of confi : 
dence 11 in the- public schools 1 ability to adequately educate their pupil s*- 
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especially in the basic skills. Increasingly, districts are realizing 
the limitations of methods of school improvement built on piecemeal 
approaches, such as untargeted . in-service training programs, or new test- 
ing programs, or adopting and implementing externally funded projects. 
These activities, however well-intentioned, simply were not reversing 
the declining test scores. " 

Some districts, such as^ our sample districts, are^now seeking more 
comprehensive and integrated approaches to developing better teachin^and 
learning. One such approach involved connecting the school district's 
testing and evaluation activities with on-going discussions about how 
to chart the district pupils' achievement, assess the effects Of various 
instructional strategies, revise those strategies, and use subsequent data 
to re-assess.' We believe this systematic approach will be increasingly 
tried by other districts. While we think that each district will have 
to evolve an approach that is appropriate to its particular context and* 
needs, it seems logical that districts beginning to consider this ap- 
proach can learn a^reat deal from the experience of these "pioneer" 
districts. They can learn of the various strategies Tfehat have been 
tried, the specific components (such as CRTs) that have been developed, 
and the kinds of barriers that have been encountered. Enlightened by 
the experience of those who have preceded them, these "newer" districts 
can, perhaps,' reduce the time atQd^cost necessary to implement such a 
system. 

Our sample districts have been deeply involved in developing these 
♦ programs and this has made it difficult for them to step back and take 
a comprehensive and somewhat detached view of their efforts. What is 
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more, they not have the opportunity to compare their efforts with 
those of other districts that are developing a similar linkage strategy. 

We see ourselves as providing two research-related services; first, 
a£ observers arjd recorders of what these districts are doing, so as to 
subsequently create from their synthesized experiences technical assis- 
tance materials for districts wishing to follow this linkage strategy 
as a means of improving pupil achievement; and second, as analysts of 
this prc&ess, we seek to understand the configuration of human, organi- 
zational, political, and technical elements that are associated with the 
implementation of this linkage strategy so as to contribute to the grow- 
ing school improvement literature. * 

With regard to oi|r technical assistance and development role, we 
realize that the linkage arrangements that ou/ sample districts are de- 
veloping are unique to each setting and that they cannot be "packaged" 
and exported to other districts. Nonetheless, there are likely portions 
of these arrangements that can provide guidance to other districts. The 
things these sample districts* have learned about the process will likely 
be of considerable interest to those who want to embark, on this strategic 
course. During the last year of this project, we will be working with 
several districts and helping them begin to design and implement such a 
program. 

With regard, to our research/analytical role, we see as a major con- 
tribution the bringing together of the research literatures frtrni several 
fields, e.g., evaluation^ testing and curriculum, and organizational 
theory as a means of gaining insights into* the dynamics of this linking 



process in school districts. Since these literatures have historically 
been developed in isolation from each other, our research provides a 
•unique vehicle for galrrFng a better understanding of their interrelation 
ships. This kind of theory/practice synthesis seems to us to be a 
necessary step i*f we are going to be able to fashion research and con- 
ceptual work into tools useful for working ofi the pressing problems 
facing public education today. 
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